Lookups by collisionless direct tables and cams

ABSTRACT

A structure and technique for preventing collisions using a hash table in conjunction with a CAM to identify and prevent a collisions of binary keys. A portion of the hash value of a binary key, which does not collide with a portion of the hash value of any other reference binary key, is used as an entry in the hash table. If two or more binary keys have identical values of the portions of the hash values, each of these binary keys are stored in their entirety, in the CAM. The key in the CAM provides a pointer to a data structure where the action associated with that binary key is stored. If the binary key is not found in the CAM, the binary key is hashed, and a specific entry in the hash table is selected using a portion of this hash value.

RELATED APLICATION

This application is a continuation of application Ser. No. 10/144,610,filed May 13, 2002.

FIELD OF THE INVENTION

This invention relates to a method and structure for preventingcollisions between two or more stored hash values of binary keys toaction items in a network environment.

BACKGROUND OF THE INVENTION

In certain networks, specific fields within message headers are used asbinary keys to search data structures for specific details regardingactions necessary for appropriate processing of those messages. Thelength of a binary key is dependent on the size of the field(s) used tocreate the key. A few example key lengths may include 32 bits for an IPaddress, 48 bits for an Ethernet MAC address, or 104 bits for a TCP/IP5-tuple. It is impractical to use these keys in their full form todirectly address corresponding entries due to the length of the keys.This can theoretically be done in content addressable memory (CAM), buttypically creates practical disadvantages because of the cost of a CAMof such size. Hence, a common approach is to hash the value of thebinary key and use a pre-selected first portion of the hashed value toaddress a specific entry in a hash table. Hashing can be accomplished bycreating a new value of the binary key having the same number of bits,which are unique to any given binary key, and then using only a portionof the bits, e.g. the first N bits to select the corresponding hashtable entry. This value is then used to address a specific entry in ahash table, sometimes referred to as a direct table DT. Either theentire hashed value or the remaining portion of the hashed value isstored in a data structure, together with the correspondingfunction-specific data denoted by the binary key. Whenever a binary keyis extracted from received messages, its value is hashed and the firstportion of the hash value is used to access an entry in the hash table.If a valid hash table entry is found, that location in the hash tablepoints to a data structure containing a complementary portion of areference hash value that is compared with the equivalent complementaryportion of the hash value generated from the message key to confirm thevalidity of the key and declare the associated action if the key is, infact, valid. This works well for some numbers; however, in some cases,the first portion of the hashed value of one binary reference key is thesame as the first portion of the hashed value of another binaryreference key. This occurs because only a portion of the newly createdvalue of the binary key is used to select an entry in the hash tableand, hence, this portion of the new value of one binary key may be thesame as that of another binary key. This is often referred to as a“collision”. In the past, this has been dealt with by the use ofpatricia tree structures or the like. But this is cumbersome andrelatively slow. Hence, a faster relatively inexpensive technique isneeded.

SUMMARY OF THE INVENTION

The present invention provides a structure and technique for totallypreventing collisions by using a hash table or direct table DT inconjunction with a content addressable memory (CAM) to identify andprevent any collisions of selected first portions of hash values, i.e.,identified first portions of different binary keys. In operation, aselected portion of the hash value of any reference binary key that doesnot collide with an identified selected portion of the hash value of anyother reference binary key is used to select an entry in the directtable. Each location addressed by the selected portion of a hash valueholds a pointer to a data structure where the action represented by thebinary key corresponding to that hashed value and the remaining portionof the hash value, or the entire hash value, are stored. However, if itis determined that two or more binary keys have identical values of theselected first portions of the hash values, each of these binary keys,or an identification specific to the key are stored, in their entirety,in the CAM, with the entry number of the matching CAM entry providing apointer to a data structure where the action associated with that binarykey is stored. Such binary keys, which are different but have the samevalue of the selected portion of their hash values, are not associatedwith entries stored in the hash table.

In operation, when a binary key is presented for search, the CAM isfirst searched to see if the binary key is stored in the CAM. If it is,the location in the CAM at which it is stored is mapped into an addresspointer to the data structure containing details regarding the action tobe performed. If the binary key is not found in the CAM, which indicatesthat there are no collisions, the binary key is hashed, and a specificentry in the hash table is selected by using the first portion of thishash value as an offset into the hash table. If the selected firstportion of the hash value accesses a valid entry in the hash table, thatentry contains a pointer to the data structure containing detailsregarding appropriate actions for processing the associated message.Either the remainder of the hashed value, or the entire hashed value, isalso stored in this data structure, so it can be compared with thehashed key constructed from the message during the process of accessingthe data structure. If the remainder or total hashed key stored in thedata structure matches the hashed key used for the search, the searchprocess has identified the desired match, and the associated actiondefined by data in the structure is indicated. If the remainder of thehash value does not compare, or if the selected portion of the hashvalue selects an invalid entry in the hash table, a no-match indicationis given, and the associated software performs appropriate defaultactions. In any event, potential collisions have been avoided, by usingconventional data tables for storing selected portions of the hashvalues of binary keys where there is no collision, and in those fewcases where there would be a collision, based on selected portions ofhash values; these are anticipated and avoided by storing the binarykeys in a CAM with pointers to the associated actions.

DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of the structure of this invention; and

FIG. 2 is a flow diagram of one search protocol according to thisinvention.

DESCRIPTION OF THE PREFERRED EMBODIMENT

FIG. 1 is a high level view of the configuration of the presentinvention. A combination of a content addressable memory (CAM) 10 and ahash table or direct table DT 12 is shown. A data structure 14 is alsoshown having a cam portion and hash table portion. Hardware 16 isprovided which will perform a hashing function of a binary key. Analternate embodiment of the invention includes software for implementingthese hash functions. In either case, software is typically used toimplement a reverse hashing required by Insert procedures to construct abinary key from the selected portions of the hashed value. A controlfunction (either hardware or software) 18 is provided to control theoperation of the CAM 10 and hashing function 16 responsive to a comparefunction 20. The underlying premise of the invention is that CAMS 10 arerelatively expensive but do function well to provide a positiveindication of a match of a binary bit number being delivered thereto. Onthe other hand, direct tables or hash tables DT 12 are relativelyinexpensive and can provide maximum storage of entries corresponding toselected segments of hashed values at a minimum cost. However, when abinary number is hashed and a selected portion of the hashed value isused to identify the entire hashed value and, hence, the binary keyvalue, there is a possibility that two different binary keys or binarynumbers will have the same value of the selected portion of the hashedvalue. Briefly, hashing, as used herein, refers to generating a numberof bits in a selected manner from the bits of a given binary number,such as a binary key, and then using a certain predetermined portion ofthe number of those bits to identify the binary key value, e.g., atypical binary key may have 32, 48, 104 bits or more, and the first Nbits are used to select an entry from the hash table, where N may belimited in practical implementations to 20 or less. A technique forproviding a hash function and reverse hash function is shown in commonlyowned application Ser. No. 09/210,222, filed Dec. 10, 1998, which isincorporated herein by reference.

In hashing, it should be understood that for any given binary key, X,there is a hash function for generating a hashed key, H(X), having thesame number of bits, x, as in the original binary key, X. H(X) may bepartitioned into two segments, h(X) and h′(X), with h(X) having a fixednumber of bits N in a specific place so that the number of bits in h(X)is greater than zero (0) and less than the total number of bits in thekey X. The segment h′(X) is the complementary function of h(X), so thath′(X) has x-N bits. Thus, h(X) concatenated with h′(X) reconstructs thehashed key H(X). Moreover, knowing both the hash function h(X) and thecomplement h′(X) allows the value X of the binary key to be recalculatedprecisely.

Thus, it is possible to have two selected first portion values which areidentical but which refer to different binary keys. Such a condition isknown as a collision, and collisions need to be avoided so that, when akey is presented for search, there will be an unambiguous pointing tothe proper action represented by a given key that is unique to the givenkey. However, the predetermined portion of hashed value could be thesame for two or more binary keys. This results in a collision that mustbe avoided in order to prevent ambiguity in an action associated withthe binary key.

In many network systems, different binary keys are typically containedin the header of a message that is being distributed within the system,and are used to guide actions taken on these messages by networkingdevices. Each binary key corresponds to attributes or details of actionsto be taken in processing a message containing the key. For example, anIP destination address may be used as a key to access data structuresidentifying the next hop address, target port to be used fortransmitting the data, transmit vs. discard indication, etc. When thisparticular key is presented within the system for search and execution,the key will be used to locate details of actions to be taken, and thesystem will take those actions based upon the particular action dataassociated with the binary key. Thus, whenever a particular key ispresented, this must be recognized as a unique binary key and a pointerdeclares the action indicated by the key.

According to the present invention, a CAM 10 is used in conjunction witha hash table 12, a data structure 14, and hardware 16 to perform ahashing and unhashing function to effectively utilize the capability ofthe CAM while minimizing its size and, thus, its cost and using thehashed value in a hash table when the CAM is not needed.

According to the present invention, a hash function accepts a binarykey, X, consisting of M bits, and computes a corresponding hashed key,H(X), that also consists of M bits. A selected first portion, h(X), ofhashed key, H(X), is used to map to a corresponding entry in a hashtable. The selected first portion, h(X), consists of N bits (where N<M).Likewise, the hash table uses an N=bit address to select one of 2^(N)entries. The output of a search is uniquely determined by the full Mbits of X or H(X). However, the first N bits (i.e. h(X)) might notcorrelate to a single unique entry. A complementary function, h′(X),consisting of M-N bits, is, therefore, defined as the remainder fromH(X), after h(X) has been segmented from it. This complementary functionis used to validate the uniqueness of an entry in the hash table viacomparison with a stored equivalent, h′(x). The invention operates asfollows. If the selected portion h(X) of the hashed value of two or morebinary keys is the same value, then each of these binary keys, or avalue unique to each key, is stored in the CAM 10, with the location ofeach CAM entry associated with the address of a corresponding datastructure containing appropriate data to guide message processingactions. The location of the data structure can be an offset value basedon the location of the matching CAM entry, or the data structure itselfcould be contained within the CAM 10, or any other technique could beused to recognize and initiate action. (The technique for insertion anddeletion of values into the CAM will be described presently.) If,however, the selected first portion h(X) of the hash value of any binarykey is unique and the binary key is not stored in the CAM 10, then theselected portion of the hash value h(X) is used to access a specificentry in the hash or direct table DT 12 at a particular location, with apointer from that location to the data structure having a correspondingaction of the binary key having that hashed value. The remainder h′(X),or the hash value H(X), is also included in the data structure. If theremainder or all of the hash value stored in the data structure matchesthe hashed key used to locate the data structure, then the action isdeclared. Thus, in operation, when a binary key is presented, acomparison is first made in the CAM 10 to see if the binary key isstored. It will be remembered that the only binary key numbers stored inthe CAM in their entirety (i.e., in their unhashed value) are thosebinary keys which have selected first portions of their hash values thatare identical to the selected first portion of some other binary key.Thus, there are a minimum number of binary keys that need to be storedin the CAM 10. If the value corresponding to the binary key is found inthe CAM 10, then a pointer from that entry points to the data structurewhere action to be taken is declared, or the location stores therequired action. If, however, the binary key is not found in the CAM 10,then the binary key is hashed and the selected portion h(X) of the hashvalue is used to access the hash table or direct table DT 12. If a validentry is found, that means that there are no other identical selectedfirst portions of the hash values, and so a pointer from that value inthe hash table or direct table DT 12 points to the data structurecontaining the remainder of the hash value and the action of the binarykey.

In a preferred embodiment, once a binary key is placed into the CAM (dueto a collision in the hash table), it will remain in the CAM even if theother colliding entries are eventually deleted via administrative tablemaintenance. A table maintenance task can manage these situations byperiodically hashing each binary key in the CAM, and searching forentries that are unique in the selected first portion h(x). Any CAMentries identified to have a unique selected first portion h(X) can thenbe added to the hash table and removed from the CAM. Those skilled inthe art will recognize that more complex implementations are possiblethat would maintain a separate data structure or an additional segmentof the base data structure to identify which CAM entries have matchingselected first portions h(x). Such additional data structures can enablethe delete process to test an entry being deleted from the CAM todetermine if the deletion would result in a remaining CAM entry that nolonger matched other entries in the selected first portion h(x), thusenabling that remaining CAM entry to be moved from the CAM into the hashtable.

Thus, the search policy can be characterized as follows where a key X ispresented for search:

The following designations are used in the description of the Searchpolicy, Insertion policy, and Deletion policy:

-   X, Y . . . =Binary Keys consisting of M bits-   H(X), H(Y)=Hash Value of Key consisting of M bits-   h(X), h(Y)=Selected Portion of Hash Value consisting of N bits-   h′(X), h′(Y)=Remainder of Hash Value consisting of (M-N) bits-   h(x), h(y)=Selected Portion of comparison Hash Value optionally    stored in data structure consisting of N bits-   h′(x), h′(y)=Remainder or Complement of comparison Hash Value stored    in data structure consisting of (M-N) bits-   A, B . . . =Action denoted by X, Y

SEARCH POLICY (FIG. 2)

-   1. Seek X in the CAM 10-   2. If X is found, then declare corresponding action A, and end.-   3. Else generate h(X) and go to corresponding hash table 12 entry.-   4. If h(X) corresponds to an invalid or empty entry, declare default    action DA, and end.-   5. If hash table entry is valid (a hit), compare remainder h′(X)    derived from search key X with h′(x) stored in data structure.-   7. If match, then declare (unambiguous and final) corresponding    action A from data structure, and end.-   8. Else, if no match of h′(X), then declare default action, and end.

In another embodiment, both the hash table 12 and CAM 10 are searchedsimultaneously. In yet another embodiment, the hash table 12 is searchedfirst rather than the CAM 10.

For insertion of an entry corresponding to a binary key number in theCAM 10 or in the hash table or direct table DT 12, the following stepsare performed. A binary key X and action A are presented for insertion.First, the M-bit binary key X is sought in the CAM 10, and if X isfound, then the system will write A over the existing action and end theprocedure. If X is not found, then X is hashed and the entry in hashtable or direct table DT 12 is accessed using an N bit addresscorresponding to h(X) to see if a valid entry is found there. If thehash table entry is invalid, then the h(X) is used to store a pointer toa data structure containing action A corresponding to X and h′(x), andthe program is ended. If an entry exists at index h(X), and contains apointer from this entry to an obsolete version of action A correspondingto X (i.e. h′(X)=h′(x)), then action data A is updated in thecorresponding data structure. However, if an entry exists at index h(X)and contains a pointer from this entry to an action B corresponding to Y(i.e. h(X)=h(y) but h′(X)˜=h′(y)), then the binary key X is entered intothe CAM 10, with the entry index corresponding to the location of a newdata structure containing action A. Following this, binary key Y isrecreated from the hash h(y), or equivalently h(X), and the complementh′(y) is stored in the data structure pointed to by the hash table entryindexed by h(X), and then binary key Y is entered into the CAM 10 withthe entry index corresponding to the location of a new data structurecontaining action B. The entry corresponding to h(Y) is deleted in thehash table 12 (i.e. marked invalid), the original data structurecontaining action B is deleted (since this data is moved to a locationcorresponding to the CAM entry index for Y), and the program is ended.

An alternate implementation includes the step of copying the pointerfrom the hash table entry to a small data portion of the new CAM entry.In this alternative, the data structure does not have to be moved, sincethe new pointer continues to point to the same data structure location.The Insertion Policy can be characterized as follows where a key X andaction A are presented for insertion.

INSERTION POLICY

-   1. Seek X in the CAM 10.-   2. If X is found, then write A over the existing action in the    corresponding data structure, and end.-   3. Else compute h(X) from X, and go to hash table 12 entry indexed    by h(X).-   4. If DT entry is unoccupied, then create a pointer from that entry    to a data structure containing index h′(x) set to a value of h′(X)    and action A, and end.-   5. Else (the entry at index h(X) has some existing entry stored with    a pointer to a data structure) compare h′(X) with h′(y) stored in    the data structure.-   6. If h′(X)=h′(y), update action data A in data structure pointed to    by hash table entry.-   7. Else (the entry at index h(X) has some existing pointer stored to    a data structure containing an action B corresponding to H(y)) enter    key X in the CAM (10). Enter action A in data structure location    corresponding to CAM entry index.-   8. Recreate Y from H(Y)=h(Y)∥h′(Y) where ∥ denotes concatenation.-   9. Enter key Y in the CAM. Enter action B in data structure location    corresponding to CAM entry index.-   10. Delete (i.e. mark invalid) entry at offset h (Y) from the hash    table. Delete original data structure holding action B.-   11. End.

For deletion, a key X is presented for deletion and the key X is soughtin the CAM 10. If X is found, then delete X and corresponding datastructure containing action A and mark for overwriting, and the programis done. Otherwise, if X is not found in the CAM 10, then thepre-selected N-bit portion h(X) of the hash value is used to index intothe hash table (DT) 12. If the hash table slot at the h(X) index isoccupied, then delete the entry and the corresponding data structurecontaining action A, mark the hash table entry as invalid, and end. Ifthe entry indexed by h(X) is invalid, or if it points to a datastructure containing h′(y), such that the compare at the end of thesearch does not match, then log a message indicating that the entrytargeted for deletion was not found, and end the program. This can bewritten as follows when a key X is presented for deletion.

DELETION POLICY

-   1. Seek X in the CAM 10.-   2. If X is found, then delete X and corresponding data structure    containing action A (mark the entry as invalid) and end.-   3. Else access entry indexed by h(X) in the hash table 12.-   4. If the hash table 12 entry indexed by h(X) is valid, then compare    h′(X) with the value of h′(x) from the corresponding data structure.-   5. If the comparison fails, the entry to be deleted is not in the    table. Log “X not found for deletion” and end.-   6. If the comparison matches, delete the hash table entry indexed by    h(X) and delete the corresponding data structure containing action    A, and end.-   7. Else (invalid entry accessed from hash table) log “X not found    for deletion” and end.

1. A method of preventing collisions between two or more binary keyswherein each binary key corresponds to an action to be taken, comprisingthe steps of: providing a hash table having a plurality of entriesindexed by a selected portion of hash values of said binary keys, eachentry pointing to a location in a data structure for storing thenon-selected portions of, or the entire hash value of, the binary keyand action data corresponding to the value of the binary key, and acontent addressable memory (CAM) having a plurality of entries, eachconfigured to store a binary key, or a value unique to a binary key, andan association to a corresponding action associated therewith; storingin said hash table a pointer to said data structure using a selected oneportion of the hash values of binary keys as an index into the hashtable when and only when said selected one portion of the hash value isnot the selected one portion of the hash value of any other binary key,and storing in the CAM binary keys or values unique to said binary keys,and establishing an association between said CAM entry locations andlocations of said data structures, when and only when the selectedportion of the hash value of any such binary key is the same as theselected portion of the hash value of one or more other binary keys. 2.The invention as defined in claim 1 further characterized by: presentinga binary key for action; searching the CAM to see if the binary key isstored in said CAM; if said binary key is found, using establishedassociation with said data structure in order to access the action dataassociated therewith; hashing the binary key and accessing an entry inthe hash table using a portion of resulting hashed binary key as anindex into said hash table to determine if the selected entry of thehash table is valid; if the entry is valid, then pointing to said datastructure in order to access the action data associated therewith. 3.The invention as defined in claim 2 wherein said CAM and said hash tableare searched simultaneously.
 4. The invention as defined in claim 2wherein said CAM is searched first, and said hash table is searched ifand only if the value of the binary key is not found in the CAM.
 5. Theinvention as defined in claim 1 wherein said binary key is storedunaltered in the CAM.
 6. The invention as defined in claim 2 whereinsaid binary key is stored unaltered in the CAM.
 7. The invention asdefined in claim 3 wherein said binary key is stored unaltered in theCAM.
 8. The invention as defined in claim 4 wherein said binary key isstored unaltered in the CAM.
 9. The invention as defined in claim 1wherein said action is stored in the CAM.
 10. The invention as definedin claim 1 wherein said action is directed by an offset value in saidCAM location.
 11. A method for preventing collisions between two or morebinary keys wherein each binary key corresponds to an action to betaken, comprising: said method including providing a hash table having aplurality of entries indexed by a selected portion of hash values ofsaid binary keys, each entry pointing to a location in a data structurefor storing the non-selected portions of, or the entire hash value of,the binary key and action data corresponding to the value of the binarykey; a content addressable memory (CAM) is provided having a pluralityof entries, each configured to store a binary key, or a value unique toa binary key, and an association to a corresponding action associatedtherewith; each entry in said hash table having an entry and a pointerto said data structure using a selected one portion of the hash valuesof binary keys as an index into the hash table when and only when saidselected one portion of hash value is not the selected one portion ofthe hash value of any other binary key using said CAM and hash table toprevent collision of any binary keys.
 12. The invention as defined inclaim 11 wherein said value stored in the CAM is the unaltered value ofeach key.
 13. The invention as defined in claim 11 wherein said actionrelated to each entry in said CAM is stored in said CAM.
 14. Theinvention as defined in claim 12 wherein said action related to eachentry in said CAM is stored in said CAM.
 15. The invention as defined inclaim 11 wherein said action related to each entry in said CAM isdesignated by an offset.
 16. The invention as defined in claim 12wherein said action related to each entry in said CAM is designated byan offset.
 17. A computer readable medium or media having thereon themethod of preventing collisions between two or more binary keys whereineach binary key corresponds to an action to be taken, comprising thesteps of: providing a hash table having a plurality of entries indexedby a selected portion of hash values of said binary keys, each entrypointing to a location in a data structure for storing the non-selectedportions of, or the entire hash value of, the binary key and action datacorresponding to the value of the binary key, and a content addressablememory (CAM) having a plurality of entries, each configured to store abinary key, or a value unique to a binary key, and an association to acorresponding action associated therewith; storing in said hash table apointer to said data structure using a selected one portion of the hashvalues of binary keys as an index into the hash table when and only whensaid selected one portion of the hash value is not the selected oneportion of the hash value of any other binary key, and storing in theCAM binary keys or values unique to said binary keys, and establishingan association between said CAM entry locations and locations of saiddata structures, when and only when the selected portion of the hashvalue of any such binary key is the same as the selected portion of thehash value of one or more other binary keys.
 18. A computer readablemedium or media having thereon the method for preventing collisionsbetween two or more binary keys wherein each binary key corresponds toan action to be taken, comprising: said method including providing ahash table having a plurality of entries indexed by a selected portionof hash values of said binary keys, each entry pointing to a location ina data structure for storing the non-selected portions of, or the entirehash value of, the binary key and action data corresponding to the valueof the binary key; a content addressable memory (CAM) is provided havinga plurality of entries, each configured to store a binary key, or avalue unique to a binary key, and an association to a correspondingaction associated therewith; each entry in said hash table having anentry and a pointer to said data structure using a selected one portionof the hash values of binary keys as an index into the hash table whenand only when said selected one portion of hash value is not theselected one portion of the hash value of any other binary key usingsaid CAM and hash table to prevent collision of any binary keys.